Method and apparatus for removing image blocking artifact by using transformation coefficient

ABSTRACT

A method and apparatus for removing an image blocking artifact by using a transformation coefficient are provided. The method includes: detecting a first blocking artifact which is in a flat region of the input image; removing the first blocking artifact using a number of low frequency coefficients from among transformation coefficients for each transformation block of a plurality of transformation blocks of the input image, based on a result of detecting the first blocking artifact; detecting an edge region of the input image; and removing a second blocking artifact, which is in an edge region of an intermediate image obtained from the removing of the first blocking artifact of the input image, based on the detected edge region in the input image.

CROSS-REFERENCE TO RELATED PATENT APPLICATION

This application claims priority from U.S. Provisional Application No.61/431,176, filed on Jan. 10, 2011, in the U.S. Patent and TrademarkOffice, and Korean Patent Application No. 10-2011-0092222, filed on Sep.9, 2011 in the Korean Intellectual Property Office, the disclosures ofwhich are incorporated herein in their entireties by reference.

BACKGROUND

1. Field

Apparatuses and methods consistent with exemplary embodiments relate toa block processing phenomenon due to image encoding, and moreparticularly, to removing image blocking artifacts generated during animage encoded process.

2. Description of the Related Art

Encoding and compression techniques are widely used to reduce datacapacity for transmitting, receiving, and storing image data, andgenerally, a block-based encoding technique is used. However, imageblocking artifacts may be generated in a restored image after decodingdue to frequency conversion and quantization, and as a result, thequality of an image may deteriorate due to horizontal and verticallattice noise.

Deblocking techniques may be used to reduce blocking artifacts generatedduring a block-based encoding process. Encoder-performed loop filtering,and post-processing filtering that does not change the structure of theencoder, are used as a common decoding technique. In the case of loopfiltering, accurate deblocking is possible using encoding informationabout each frame. However, if deblocking is performed by an encodingsystem, encoding complexity will increase, as well as channel bandwidthusage. In the case of post-processing filtering, deblocking filtering isperformed by a decoding terminal independently from an encoding system,after restoring an image. Consequently, calculation load of the encodingsystem is not increased. However, since deblocking is performedindependently from the encoding and decoding processes, and afterdecoding is completed, encoding information used during the decodingprocess of an image is difficult to acquire and use during thedeblocking operation.

SUMMARY

One or more exemplary embodiments may overcome the above disadvantagesand other disadvantages not described above. However, one or moreexemplary embodiments are not required to overcome the disadvantagesdescribed above, and may not overcome any of the problems describedabove.

According to an aspect of an exemplary embodiment there is provided amethod of removing blocking artifacts in an input image, the methodincluding: detecting a first blocking artifact, which is in a flatregion of the input image; removing the first blocking artifact using apredetermined number of low frequency coefficients from amongtransformation coefficients for each transformation block of a pluralityof transformation blocks of the input image, based on the detecting ofthe first blocking artifact; detecting an edge region of the inputimage; and removing a second blocking artifact, which is in an edgeregion of an intermediate image obtained from the removing of the firstblocking artifact of the input image, based on the detected edge regionin the input image.

The detecting of the first blocking artifact in the flat region mayinclude: determining flat blocks in the input image; and detecting thefirst blocking artifact based on a blocking artifact strength of theflat blocks.

The first blocking artifact based on the blocking artifact strength ofthe flat blocks may include determining a current blocking artifactstrength of a current flat block based on a difference between directcurrent coefficients of the current flat block and adjacent blocks fromamong the flat blocks.

The detecting of the first blocking artifact based on the blockingartifact strength of the flat blocks may include: determining localblocking artifact intensities of the flat blocks; determining a globalblocking artifact strength of the input image based on an average of thelocal blocking artifact intensities of the flat blocks; and determiningthat the blocking artifact is generated in the flat region of the inputimage if the global blocking artifact strength exceeds a predeterminedthreshold value.

The removing of the first blocking artifact may include: determining thetransformation coefficients according to pixels of the input image bytransforming pixels in a block unit including a current pixel and havinga predetermined size, for each of the pixels of the input image; andcompensating for the predetermined number of low frequency coefficientsfrom among the transformation coefficients according to pixels, for theeach of the pixels of the input image.

The removing of the flat block artifact in the flat region may include:replacing the transformation coefficients according to blocks, which areobtained by dividing the input image in units of the transformationblocks and transforming pixels in each of the transformation blocks,with the compensated for low frequency coefficients; and generating anintermediate image in a spatial region by inverse transforming thetransformation coefficients of the input image according to thetransformation blocks.

The compensating for of the predetermined number of low frequencycoefficients may include: generating low frequency coefficient images,in which low frequency coefficients from among the predetermined numberof low frequency coefficients which are in a same coefficient locationare arranged in an order of a corresponding block, for coefficientlocations of the low frequency coefficients, by classifying thepredetermined number of low frequency coefficients according tocoefficient locations in the transformation blocks; and removing thefirst blocking artifact for each of the low frequency coefficient imagesgenerated according to the coefficient locations in the transformationblocks.

The replacing of the transformation coefficients according to blocks,which are obtained by dividing the input image in block units with thecompensated for low frequency coefficients, may include: extracting lowfrequency coefficients of blocks in a same location as thetransformation blocks of the input image, from among the compensated forlow frequency coefficients; and replacing low frequency coefficients ina same location as the extracted low frequency coefficients, from amongthe low frequency coefficients of the transformation blocks of the inputimage, with the extracted low frequency coefficients.

The first blocking artifact may include determining a smoothingfiltering strength for removing the first blocking artifact, for each ofthe flat blocks, based on the local blocking artifact intensities of theflat blocks, and removing the first blocking artifact based on thedetermined smoothing filtering strength.

The first blocking artifact may include determining a filtering strengthof total variation filtering for removing the first blocking artifactbased on the local blocking artifact intensities of the flat blocks, andremoving the first blocking artifact by performing the total variationfiltering based on the determined filtering strength.

The detecting of the edge region in the input image may include:generating a low frequency band image by inverse transforming an imagethat includes the predetermined number of low frequency coefficients fortransformation blocks of the input image, wherein other coefficients are0; and generating a high frequency band image by subtracting the lowfrequency band image from the input image.

The detecting of the edge region in the input image may further includedetermining a transformation block of which the energy of samples ishigher than a predetermined threshold value from among transformationblocks of the high frequency band image, as an edge block.

The energy of samples may be an average of absolute values of thetransformation coefficients of the transformation blocks or an averageof absolute values of pixel values of the transformation blocks.

The removing of the second blocking artifact may include: grouping edgeblocks forming a same edge, from among edge blocks of the edge region ofthe intermediate image, in correspondence to the edge region of theinput image; performing a three-dimensional (3D) orthogonaltransformation on the edge blocks in each group resulting from thegrouping; and removing the second blocking artifact by using 3Dtransformation coefficients of the edge blocks in each group resultingfrom the grouping.

The removing of the blocking artifact of the edge blocks using the 3Dtransformation coefficients of the edge blocks in each group may includeperforming threshold filtering for transforming the 3D transformationcoefficients to 0 if the 3D transformation coefficients are less than apredetermined threshold value.

The removing of the blocking artifact of the edge blocks using the 3Dtransformation coefficients of the edge blocks in each group mayinclude: performing threshold filtering on the 3D transformationcoefficients; restoring the edge blocks in each group by performing aninverse 3D orthogonal transformation on the 3D transformationcoefficients on which the threshold filtering is performed; and updatingthe edge region detected in the intermediate image based on the restorededge blocks.

The updating of the edge region detected in the intermediate image basedon the restored edge blocks may include updating the edge region bydetermining a weighted average block with respect to edge blocks in anoverlapping region from among edge blocks restored by performing the 3Dorthogonal transform, the threshold filtering, and the inverse 3Dorthogonal transformation according to a plurality of groups.

The method may further include receiving the input image, wherein theinput image is decoded and restored by a decoder.

According to an aspect of an exemplary embodiment there is provided anapparatus for removing image blocking artifacts in an input image, theapparatus including: a flat region blocking artifact detector thatdetects a first blocking artifact that is in a flat region of the inputimage; a flat region blocking artifact remover that removes the firstblocking artifact by using a predetermined number of low frequencycoefficients from among transformation coefficients according to blocks,for each of transformation blocks of the input image based on the resultof detecting; an edge region detector that detects an edge region in theinput image; and an edge region blocking artifact remover that removes asecond blocking artifact, which is in an edge region of an intermediateimage, obtained by removing the first blocking artifact of the inputimage, based on the detected edge region in the input image.

The flat region blocking artifact detector may determine flat blocks inthe input image, and detect the first blocking artifact based on ablocking artifact strength of the flat blocks.

The flat region blocking artifact detector may determine a currentblocking artifact strength of a current flat block based on a differencebetween direct current coefficients of the current flat block andadjacent blocks from among the flat blocks.

The flat region blocking artifact detector may: determine local blockingartifact intensities of the flat blocks; determine a global blockingartifact strength of the input image based on an average of the localblocking artifact intensities of the flat blocks; and determine that theblocking artifact is generated in the flat region of the input image ifthe global blocking artifact strength exceeds a predetermined thresholdvalue.

The flat region blocking artifact remover may determine transformationcoefficients according to pixels of the input pixels by transformingpixels in a block unit including a current pixel and having apredetermined size, for each of the pixels of the input image, andcompensate for the predetermined number of low frequency coefficientsfrom among the transformation coefficients for the each of the pixels ofthe input image.

The flat region blocking artifact remover may replace the transformationcoefficients according to blocks with the compensated for low frequencycoefficients, for the transformation blocks of the input image, andgenerate an intermediate image in a spatial region by inversetransforming the transformation coefficients of the input image for eachof the transformation blocks.

The flat region blocking artifact remover may generate low frequencycoefficient images, in which low frequency coefficients from among thepredetermined number of low frequency coefficients which are in a samecoefficient location are arranged in an order of a corresponding block,for coefficient locations of the low frequency coefficients, byclassifying the predetermined number of low frequency coefficientsaccording to coefficient locations in the transformation blocks, andremove the first blocking artifact in flat blocks for each of the lowfrequency coefficient images generated according to the coefficientlocations.

The flat region blocking artifact remover may extract low frequencycoefficients of blocks in a same location as the transformation blocksof the input image, from among the compensated for low frequencycoefficients, and replace low frequency coefficients in a same locationas the extracted low frequency coefficients, from among the lowfrequency coefficients of the transformation blocks of the input image,with the extracted low frequency coefficients.

The flat region blocking artifact remover may determine a smoothingfiltering strength for removing the blocking artifact according to theflat blocks based on the local blocking artifact intensities of the flatblocks, and remove the first blocking artifact based on the determinedsmoothing filtering strength.

The flat region blocking artifact remover may determine a filteringstrength of total variation filtering for removing the first blockingartifact based on the local blocking artifact intensities of the flatblocks, and remove the first blocking artifact by performing the totalvariation filtering based on the determined filtering strength.

The edge region detector may generate a low frequency band image byinverse transforming an image that includes the predetermined number oflow frequency coefficients for transformation blocks of the input image,wherein other coefficients are 0, and generate a high frequency bandimage by subtracting the low frequency band image from the input image.

The edge region detector may determine a transformation block of whichthe energy of samples is higher than a predetermined threshold valuefrom among transformation blocks of the high frequency band image, as anedge block.

The energy of samples may be an average of absolute values of thetransformation coefficients of the transformation blocks or an averageof absolute values of pixel values of the transformation blocks.

The edge region blocking artifact remover may group edge blocks forminga same edge, from among edge blocks of the edge region of theintermediate image, in correspondence to the edge region of the inputimage, perform a three-dimensional (3D) orthogonal transformation on theedge blocks in each group grouped by the edge region blocking artifactremover, and remove the blocking artifact of the edge blocks by using 3Dtransformation coefficients of the edge blocks in each group grouped bythe edge region blocking artifact remover.

The edge block blocking artifact remover may perform threshold filteringfor transforming the 3D transformation coefficients to 0 if the 3Dtransformation coefficients are less than a predetermined thresholdvalue.

The edge block blocking artifact remover may perform threshold filteringon the 3D transformation coefficients; restores the edge blocks in eachgroup by performing an inverse 3D orthogonal transformation on the 3Dtransformation coefficients on which the threshold filtering isperformed, and updates the edge region detected in the intermediateimage based on the restored edge blocks.

The edge block blocking artifact remover may update the edge region bydetermining a weighted average block with respect to edge blocks in anoverlapping region from among edge blocks restored by performing the 3Dorthogonal transform, the threshold filtering, and the inverse 3Dorthogonal transformation according to a plurality of groups.

The input image may be decoded and restored by a decoder.

An aspect of an exemplary embodiment provides a computer-readablerecording medium having recorded thereon a program for executing amethod of removing blocking artifacts in an input image, the methodincluding: detecting a first blocking artifact, which is in a flatregion of the input image; removing the first blocking artifact using apredetermined number of low frequency coefficients from amongtransformation coefficients for each transformation block of a pluralityof transformation blocks of the input image, based on the detecting ofthe first blocking artifact; detecting an edge region of the inputimage; and removing a second blocking artifact, which is in an edgeregion of an intermediate image obtained from the removing of the firstblocking artifact of the input image, based on the detected edge regionin the input image.

According to an aspect of an exemplary embodiment there is provided amethod of removing blocking artifacts in an input image, the methodincluding: detecting at least one first blocking artifact, which is in aflat region of the input image, based on a first threshold comparison;transforming blocks of the input image, wherein the transforminggenerates transformation coefficients for each transformed block of theinput image; extracting a predetermined number of low frequencycoefficients from among the transformation coefficients generated fromthe transforming; generating low frequency coefficient images based onthe extracted predetermined number of low frequency coefficients;generating compensated low frequency coefficient images by filtering theat least one first blocking artifact from the generated low frequencycoefficient images; extracting compensation coefficients from thecompensated low frequency coefficient images; and generating anintermediate image by inverse transforming the compensated low frequencycoefficient images.

The method may further include: detecting at least one edge block in theintermediate image based on a second threshold comparison; performing athree-dimensional (3D) transformation on groups of edge blocks, whichform a same edge, from among the at least one edge block detected in theintermediate image; filtering 3D transformation coefficients resultingfrom the 3D transformation; performing an inverse 3D transformationbased on a result of the filtering of the 3D transformationcoefficients; and generating a final image based on the performing ofthe inverse 3D transformation.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features will become more apparent by describing indetail exemplary embodiments with reference to the attached drawings inwhich:

FIG. 1 is a block diagram of an apparatus for removing an image blockingartifact by using a transformation coefficient, according to anexemplary embodiment;

FIG. 2 is a flow diagram for describing a related art method of removinga blocking artifact;

FIG. 3A is a block diagram of an apparatus for removing an imageblocking artifact, according to another exemplary embodiment;

FIGS. 3B and 3C are block diagrams respectively of apparatuses forremoving an image blocking artifact, according to other exemplaryembodiments;

FIG. 4 is a diagram for describing a method of detecting a blockingartifact in a flat region, according to an exemplary embodiment;

FIG. 5 is a diagram for describing a method of removing a blockingartifact in a flat region, according to an exemplary embodiment;

FIG. 6 illustrates graphs for describing filtering for removing ablocking artifact in a flat region, according to an exemplaryembodiment;

FIG. 7 is a diagram for describing a method of removing a blockingartifact in an edge region, according to an exemplary embodiment; and

FIG. 8 is a flowchart illustrating a method of removing an imageblocking artifact by using a transformation coefficient, according to anexemplary, embodiment.

DETAILED DESCRIPTION

Hereinafter, exemplary embodiments will be described more fully withreference to the accompanying drawings.

FIG. 1 is a block diagram of an apparatus 100 for removing an imageblocking artifact by using a transformation coefficient, according to anexemplary embodiment.

The apparatus 100 includes a flat region blocking artifact detector 110,a flat region blocking artifact remover 120, an edge region detector130, and an edge region blocking artifact remover 140.

The apparatus 100 is an image processing processor, and may beinterlocked with a calculation processor such as a graphic processingunit (GPU), or controlled by a calculation processor. Also, theapparatus 100 may be installed in a decoder or as an external device ofa decoder. More generally, parts of the apparatus 100 may be realized byhardware components, such as by processor(s) or circuit(s), and/or bysoftware components executed by hardware components.

The flat region blocking artifact detector 110 detects a blockingartifact in a flat region of an input image of the apparatus 100. Theinput image of the apparatus 100 may be an image obtained by decoding anencoded image using a decoder and then restoring the decoded image in aspatial region.

If the blocking artifact is detected in the flat region based on theresult of the detecting by the flat region blocking artifact detector110, the flat region blocking artifact remover 120 removes the blockingartifact in the flat region by using a predetermined number of lowfrequency coefficients from among transformation coefficients, forblocks of the input image.

The edge region detector 130 detects an edge region of the input imagein comparison with the flat region detected by the flat region blockingartifact detector 110.

The edge region blocking artifact remover 140 removes a blockingartifact in the edge region detected by the edge region detector 130.

In detail, the flat region blocking artifact detector 110 according toan exemplary embodiment may determine flat blocks in the input image,and detect a blocking artifact in the flat region based on blockingartifact intensities of the flat blocks. According to an exemplaryembodiment, a blocking artifact strength (BAS) of a current flat blockmay be determined based on a difference between direct current (DC)coefficients of the current flat block and adjacent blocks.

The flat region blocking artifact remover 120 according to an exemplaryembodiment may determine transformation coefficients for pixels of theinput image by performing a transformation on pixels in a block unitincluding a current pixel and having a predetermined size, for eachpixel of the input image. Examples of a block transformation accordingto an exemplary embodiment may include a discrete cosine transformation(DCT), an orthogonal transformation, an integer transformation, etc. Theflat region blocking artifact remover 120 according to an exemplaryembodiment may remove a blocking artifact in a flat block beingprocessed by compensating for a predetermined number of low frequencycoefficients from among transformation coefficients, for each pixel ofthe pixels of the input image.

Each of the unit blocks of pixels of the input image includes a currentpixel and corresponding pixels in a current block having a predeterminedsize. Thus blocks of pixels of the input image overlap if they areblocks of adjacent pixels, such as shown in the example of FIG. 5.However, transformation blocks do not overlap since they are obtained bydividing the input image into data blocks of a predetermined size forperforming a block transformation of the entire input image.

The flat region blocking artifact remover 120 according to an exemplaryembodiment may (a) determine a smoothing filtering strength of theblocking artifact of the flat blocks based on the blocking artifactintensities of the flat blocks, and (b) remove the blocking artifact inthe flat block by performing smoothing filtering on a block boundary ofthe flat blocks based on the determined smoothing filtering strength.

The flat region blocking artifact remover 120 according to an exemplaryembodiment may determine a filtering strength of a total variationfiltering for removing the blocking artifact in the flat region, basedon local blocking artifact intensities of the flat block. Accordingly,the flat region blocking artifact remover 120 may remove the blockingartifact of the flat block by performing a total variation filteringbased on intrinsic smoothing filtering strength determined for each flatblock. Accordingly, an output image of the flat region blocking artifactremover 120 may be an intermediate image obtained by removing theblocking artifact in the flat region of the input image.

The edge region detector 130 according to an exemplary embodiment maygenerate a low frequency band image by inverse transforming an imagethat includes a predetermined number of low frequency coefficients fromamong the transformation coefficients, wherein other coefficients are 0,for the transformation blocks of the input image. The edge regiondetector 130 according to an exemplary embodiment may also generate ahigh frequency band image by subtracting the low frequency band imagefrom the input image.

The edge region detector 130 according to an exemplary embodiment maydetermine a transformation block of which the energy of the samples ishigher than a predetermined threshold value from among transformationblocks of the high frequency band image as an edge block. The energy ofthe samples may be an average of absolute values of the transformationcoefficients of the transformation blocks or an average of absolutevalues of the pixel brightness of pixels of the transformation blocks.

The edge region blocking artifact remover 140 according to an exemplaryembodiment receives the intermediate image from which the blockingartifact in the flat region is removed by the flat region blockingartifact remover 120. The edge region blocking artifact remover 140 maydetermine an edge region of the intermediate image, which corresponds tothe edge region detected by the edge region detector 130. For example,the edge region blocking artifact remover 140 may determine edge blocksin the same location as edge blocks determined by the edge regiondetector 130, from among transformation blocks of the intermediateimage.

The edge region blocking artifact remover 140 according to an exemplaryembodiment may group edge blocks forming the same edge from among theedge blocks of the intermediate image, and perform a three-dimensional(3D) orthogonal transformation on edge blocks in a group for each of thegroups. The edge region blocking artifact remover 140 according to anexemplary embodiment may remove blocking artifact in the edge blocks byusing a 3D orthogonal transformation coefficients of the edge blocks ina group, according to the groups.

The edge region blocking artifact remover 140 according to an exemplaryembodiment may perform threshold filtering on the 3D transformationcoefficients of the edge blocks in a group. The edge region blockingartifact remover 140 according to an exemplary embodiment may restorethe edge blocks in a group according to groups by performing inverse 3Dorthogonal transformation on the 3D transformation coefficients on whichthe threshold filtering is performed.

The edge region blocking artifact remover 140 according to an exemplaryembodiment may update the detected edge region of the input image basedon the restored edge blocks. In other words, the edge region blockingartifact remover 140 may update the edge blocks of the intermediateimage from which the blocking artifact in the flat region is removedsuch that the blocking artifact of the edge blocks is additionallyremoved.

Accordingly, the apparatus 100 first detects and removes the blockingartifact in the flat region and then removes the blocking artifact inthe edge region, thereby naturally removing the image blocking artifactgenerated through block transformation of an image. Also, the apparatus100 may remove the image blocking artifact by only using imageinformation of the input image.

If the input image is an encoded image restored via decoding, theapparatus 100 may remove the image blocking artifact by only using therestored image, i.e., without having to use encoding informationdetermined during an encoding process. Specifically, since most videodata uploaded to a website is already compressed, the video data mayhave low quality compared to the original data, and it may be difficultto obtain encoding information. The apparatus 100 is able to estimatethe BAS by using a deblocking technique without using various types ofencoding information (such as the size of a transformation block in aframe, a quantization parameter (QP) and a quantization table), and toreduce a blocking artifact, without damaging the quality of an image,based on estimated filtering strength.

FIG. 2 is a flow diagram for describing a related art method of removinga blocking artifact.

According to a related art post-processing filtering technique forreducing a blocking artifact, one-dimensional (1D) filtering isperformed on a block boundary between adjacent blocks based on BAS.

For example, horizontal block boundary strength (HBS) may be used as theBAS of a transformation block 200 having a size of 8×8.

The HBS of the transformation block 200 may be set to 1 if onlycoefficients 210 at the first left column from among DCT coefficientsare not 0 The HBS of the transformation block 200 may be set to 0 ifthere is a coefficient that is not 0 from among coefficients excludingthe coefficients 210.

When blocks A through F are consecutively arranged in a horizontaldirection, the HBS may be determined with respect to a block boundary220 between a block group J including the blocks A through C and a blockgroup I including the blocks D through F.

For example, a method of determining the BAS includes operations 230through 260. In operation 230, HBS_(I) of the block group I and HBS_(J)of the block group J are determined.

If the HBS₁ and the HBS are both 1 (i.e., HSB_(I)=1 & HSB_(J)=1), it maybe determined that the block boundary 220 is strong, and thus 7-tabfiltering is performed on the block boundary 220 in operation 240. Inother words, 6 pixel values arranged in the horizontal direction may becompensated for based on the block boundary 220.

Alternatively, if at least one of the group consisting of the HSB_(I)and the HSB_(C) is not 1 (i.e., !(HSB_(I)=1 & HSB_(J)=1)), a difference(|D−C|) between pixel values at the block boundary 220 of the adjacentblocks D and C and a QP are compared in operation 250. If the differenceis less than the QP, it is determined that the block boundary 220 isweak, and thus a smoothing filtering which is weaker than 7-tabfiltering is performed on the block boundary 220 in operation 260. Forexample, 4 pixel values are compensated for.

Since filtering strength is determined by the QP in the related artpost-processing filtering technique described with reference to FIG. 2,encoding information is required. Also, since filtering is performed onpixels of a line perpendicular to the block boundary 220 and smoothingfiltering is performed by changing the filtering strength line-by-line,continuity between pixels in a horizontal direction of the blockboundary 200 may be damaged.

Also, since the related art post-processing filtering technique onlyprocesses a signal in a straight line by using 1D filtering, smoothingfiltering cannot be performed by considering a two-dimensional (2D)structure of the signal. For example, if an edge exists at a blockboundary, the edge may appear unnaturally cut due to the performing ofthe block transformation. A 1D filtering operation flattens the blockboundary without preserving the cut edge.

Further, the related art post-processing filtering technique uses a QPto determine filtering strength, but in an actual moving imagecompressing technique, different quantization tables are used accordingto the image frames, and the QP may vary according to the blocks in aframe. Thus, it is not efficient to actually use a quantization tableduring a post-processing operation.

FIG. 3A is a block diagram of an apparatus 300 for removing an imageblocking artifact, according to another exemplary embodiment.

The apparatus 300 according to the current exemplary embodiment includesa flat region blocking artifact detector 110, a flat region blockingartifact remover 120, an edge region detector 130, and an edge regionblocking artifact remover 140, and has the same basic structure as theapparatus 100 of FIG. 1. However, the flat region blocking artifactdetector 110 according to the current exemplary embodiment may include aflat region BAS determiner 310 and a filtering determiner 320. Also, theedge region detector 130 according to the current exemplary embodimentmay include a high band (HB) image generator 330 and an edge regiondetector 340.

In detail, the flat region blocking artifact detector 110 may determineflat blocks in an input image, determine the BAS in the flat blocks, anddetect a blocking artifact in a flat region based on the BAS in the flatblocks.

The flat region BAS determiner 310 may determine the BAS of a currentflat block based on a difference between DC coefficients of adjacentblocks of the current flat block. The flat region BAS determiner 310 maydetermine individual BASes of the flat blocks, i.e., local BASes of theflat blocks.

The flat region BAS determiner 310 may determine a global BAS of theinput image based on an average of the local BASes of the flat blocks.

A method of determining a global BAS performed by the flat region BASdeterminer 310 is described below with reference to FIG. 4.

The filtering determiner 320 may determine whether a blocking artifactis generated in the flat region of the input image by comparing theglobal BAS determined by the flat region BAS determiner 310 and apredetermined threshold value. Along with the result of determiningwhether the blocking artifact is generated in the flat region of theinput image, the filtering determiner 320 may determine whetherdeblocking filtering is required to remove the blocking artifact of theinput image.

In other words, if it is determined that the global BAS exceeds apredetermined threshold value, the filtering determiner 320 maydetermine that the blocking artifact is generated in the flat region ofthe input image, and determine to perform deblocking filtering forremoving the blocking artifact in the flat region of the input image.

However, if it is determined that the global BAS does not exceed thepredetermined threshold value, the filtering determiner 320 may outputthe input image as a final result image without performing deblockingfiltering on the input image.

The flat region blocking artifact remover 120 may determinetransformation coefficients for the input image pixels by performing,for each pixel, a transformation on pixels in a block unit, including acurrent pixel and having a predetermined size, and remove a blockingartifact in the flat block by compensating for a predetermined number oflow frequency coefficients from among the transformation coefficientsaccording to the input image pixels, for each pixel.

The flat region blocking artifact remover 120 may replace transformationcoefficients in the same location as the predetermined number ofcompensated for low frequency coefficients, from among thetransformation coefficients of the transformation blocks of the inputimage, with the compensated for low frequency coefficients. The flatregion blocking artifact remover 120 may extract low frequencycoefficients in blocks in the same location as the transformation blocksof the input image, from among the compensated for low frequencycoefficients. The flat region blocking artifact remover 120 may replacelow frequency coefficients in the same location as the extracted lowfrequency coefficients, from among the low frequency coefficients of thetransformation blocks of the input image, with the extracted lowfrequency coefficients.

The flat region blocking artifact remover 120 may generate anintermediate image in a spatial region by inverse transforming thetransformation coefficients of the input image, wherein sometransformation coefficients are replaced with the compensated for lowfrequency coefficients, according to transformation blocks. Thegenerated intermediate image may be an image obtained by removing theblocking artifact in the flat region from the input image.

The flat region blocking artifact remover 120 may separately compensatefor the predetermined number of low frequency coefficients according tocoefficient locations. For example, the flat region blocking artifactremover 120 may classify the predetermined number of low frequencycoefficients according to coefficient locations in the transformationblocks, and generate low frequency coefficient images in which lowfrequency coefficients in the same coefficient locations are arranged inan order of a corresponding block, for the coefficient locations of thelow frequency coefficients.

The flat region blocking artifact remover 120 may remove the blockingartifact in the flat region by using the generated low frequencycoefficient images.

For example, the flat region blocking artifact remover 120 may determinesmoothing filtering strength of the blocking artifact in the flat blockbased on a BAS of the flat block for each low frequency coefficientimage, and remove the blocking artifact in the flat block based on thedetermined smoothing filtering strength. Alternatively, the flat regionblocking artifact remover 120 may determine filtering strength of totalvariation filtering for each flat block based on the local BASes of theflat blocks for each low frequency coefficient image.

Accordingly, the flat region blocking artifact remover 120 maycompensate for the low frequency coefficients by performing totalvariation filtering based on intrinsic smoothing filtering strengthdetermined according to flat regions, for each low frequency coefficientimage.

The flat region blocking artifact remover 120 may replace transformationcoefficients in the same location as the low frequency coefficientscompensated for by using the low frequency coefficient images, fromamong the transformation coefficients of the transformation blocks ofthe input image, with the compensated for low frequency coefficients.The flat region blocking artifact remover 120 may extract the lowfrequency coefficients in the locations corresponding to thetransformation blocks of the input image, from among the compensated forlow frequency coefficients according to the low frequency coefficientimages.

In other words, when the low frequency coefficient images are generated,low frequency coefficients in a predetermined coefficient location areextracted for blocks according to the pixels of the input image and arearranged according to an order of the pixels of the input image.Accordingly, the size of the low frequency coefficient images isidentical to the size of the input image, and by performing filteringfor removing the blocking artifacts in the flat blocks of the lowfrequency coefficient images, the transformation coefficients of thecompensated for low frequency coefficient images correspond to allpixels of the input image in a one-to-one manner.

Accordingly, the flat region blocking artifact remover 120 may extracttransformation coefficients in locations corresponding to thetransformation blocks of the input image, from among the transformationcoefficients of the compensated for low frequency coefficient images,and replace the transformation coefficients in the predeterminedcoefficient location in the transformation blocks of the input imagewith the transformation coefficients extracted from the compensated forlow frequency coefficient images. The flat region blocking artifactremover 120 may replace the transformation coefficients in thecoefficient locations in the transformation blocks of the input imagewith the transformation coefficients extracted from the compensated forlow frequency coefficient images, in correspondence to coefficientlocations of low frequency coefficients forming each low frequencycoefficient image, which occupy the blocks according to pixels of theinput image.

The removing of the blocking artifact in the flat region by using thelow frequency coefficient images generated according to coefficientlocations of the low frequency coefficients is described in detail belowwith reference to FIGS. 5 and 6.

The HB image generator 330 of the edge region detector 130 may generatea low frequency band image by inverse transforming an image formed oftransformation blocks including a predetermined number of low frequencycoefficients from among transformation coefficients according to thetransformation blocks of the input image, wherein other coefficients are0. The HB image generator 330 may generate a high frequency band imageby subtracting the low frequency band image including only thepredetermined number of low frequency coefficients from thetransformation blocks of the input image.

The HB image generator 330 may generate a high frequency band image bysubtracting the flat regions detected by the flat region blockingartifact detector 110 from the transformation blocks of the input image.

The edge region detector 340 of the edge region detector 130 maydetermine a transformation block of which the energy of samples ishigher than a predetermined threshold value, from among transformationblocks of the high frequency band image, as an edge block. The energy ofsamples may be an average of absolute values of the transformationcoefficients of the transformation blocks or an average of absolutevalues of pixels of the transformation blocks.

The edge region blocking artifact remover 140 receives the intermediateimage from which the blocking artifact in the flat region is removed bythe flat region blocking artifact remover 120. The edge region blockingartifact remover 140 may determine an edge region of the intermediateimage in correspondence to the edge region detected by the edge regiondetector 130. For example, the edge region blocking artifact remover 140may determine edge blocks in the same location as the edge blocksdetermined by the edge region detector 130, from among thetransformation blocks of the intermediate image.

The edge region blocking artifact remover 140 may group edge blocksforming the same edge from among the edge blocks of the intermediateimage, and perform a 3D orthogonal transformation with respect to edgeblocks in a group, for each of the groups. The edge region blockingartifact remover 140 may remove a blocking artifact in the edge blocksby using 3D transformation coefficients of the edge blocks in a group,for each of the groups.

The edge region blocking artifact remover 140 may perform 3D orthogonaltransformation on blocks grouped via block matching, and remove ablocking artifact in a group of edge blocks forming the same edge byusing a block-matching and 3D filtering (BM3D) technique for performingfiltering on 3D transformation coefficients.

The edge region blocking artifact remover 140 may perform a thresholdfiltering on the 3D transformation coefficients of edge blocks in agroup. For example, the edge region blocking artifact remover 140 mayperform threshold filtering of transformation coefficients to “0” on theedge blocks in a predetermined group if the 3D transformationcoefficients are less than a predetermined threshold value.

The edge region blocking artifact remover 140 may restore the edgeblocks in a group according to the groups by performing inverse 3Dorthogonal transformation on the 3D transformation coefficients on whichthe threshold filtering is performed.

The edge region blocking artifact remover 140 may restore a plurality ofedge blocks by performing a 3D orthogonal transformation, a thresholdfiltering, and an inverse 3D orthogonal transformation according to thegroups, and update the edge region by determining a weighted averageblock with respect to overlapping edge blocks of the restored edgeblocks, for the groups.

The edge region blocking artifact remover 140 may update the edge regionin the intermediate image based on the restored edge blocks. Since theedge region blocking artifact remover 140 updates the edge region fromwhich a blocking artifact in the edge blocks are additionally removed,from the intermediate image from which the blocking artifact in the flatregion is removed, an image from which a blocking artifact generated dueto block transformation may finally be output.

The method of removing the blocking artifact in the edge region via 3Dtransformation on the groups of edge blocks will be described below indetail with reference to FIG. 7.

FIGS. 3B and 3C are block diagrams of apparatuses 350 and 370,respectively, for removing an image blocking artifact, according toother exemplary embodiments.

The apparatuses 350 and 370 according to the current exemplaryembodiments include a flat region blocking artifact detector 110, a flatregion blocking artifact remover 120, an edge region detector 130, andan edge region blocking artifact remover 140, and basically has the samestructure as the apparatus 100 of FIG. 1.

Similar to operations of the apparatus 100, the flat region blockingartifact detector 110 detects a blocking artifact in a flat region of aninput image; the flat region blocking artifact remover 120 removes theblocking artifact in the flat region by using a predetermined number oflow frequency coefficients from among transformation coefficients, forblocks of the input image; the edge region detector 130 detects an edgeregion of the input image; and the edge region blocking artifact remover140 removes a blocking artifact in the edge region.

In case of the apparatus 350, the edge region detector 130 may detectedge regions of the input image, prior to operations for detectingblocking artifacts via the flat region blocking artifact detector 110.Thus, the flat region blocking artifact detector 110 may receive resultsof the detecting of the edge regions from the edge region detector 130,and detect blocking artifacts in flat regions of an image which may benot the detected edge regions.

In case of the apparatuses 370, the flat region blocking artifactdetector 110 and the edge region detector 130 may respectively detectflat regions and edge regions of the input image.

FIGS. 3A, 3B and 3C, illustrate examples where the blocking artifacts ofthe edge regions and flat regions may be separately removed. The flatregion blocking artifact remover 120 may remove block artifacts of theflat regions detected by the flat region blocking artifact detector 110,and then the edge region blocking artifact remover 140 may furtherremove blocking artifacts of edge regions from an image in which blockartifacts of flat regions are removed.

FIG. 4 is a diagram for describing a method of detecting a blockingartifact in a flat region 410, according to an exemplary embodiment.Detecting of a blocking artifact in a flat region performed by the flatregion blocking artifact detector 110 is described with reference toFIG. 4.

The apparatuses 100 and 300 according to the exemplary embodiments mayremove a blocking artifact in a flat region by using low frequencycoefficients from among transformation coefficients of a transformationblock. Specifically, a blocking artifact where quantization of DCcoefficients from among the low frequency coefficients is severe may begenerated. Accordingly, the flat region blocking artifact detector 110may determine the BAS by using a difference between DC coefficients ofadjacent blocks, i.e., a gradient of DC coefficients of adjacent blocks.

DC coefficients 411 through 426 are extracted from transformationcoefficients of flat blocks F1 through F16 of a transformation region inthe flat region 410 of the input image, and are merged to form an imageI_(DC) 430.

The flat region blocking artifact detector 110 may determine a local BASfor locations of flat blocks by using a gradient of the image 430. Theflat region blocking artifact detector 110 may determine a global BAS bycalculating an average of the local BASes of all flat blocks of the flatregion 410. In order to determine the global BAS (BASg), an average ofabsolute values of gradients in horizontal and vertical directions ofthe image I_(DC) 430, including only the DC coefficients of all flatblocks, may be calculated.

According to Equation 1 (given below), a local BAS of each flat blockwith respect to N flat blocks F of the flat region 410 of the inputimage may be a sum of an absolute value |∇_(x)I_(DC)| of a gradient inan x-axis direction and an absolute value |∇_(y)I_(DC)| of a gradient inan y-axis direction for each transformation coefficient of the imageI_(DC) 430. Accordingly, the BASg of the flat region 410 is representedby Equation 1, i.e., an average of local BASes of the transformationcoefficients of the image I_(DC) 430.

$\begin{matrix}{{BAS}_{g} = {\frac{1}{N}{\sum\limits_{F}\left( {{{\nabla_{x}I_{D\; C}}} + {{\nabla_{y}I_{D\; C}}}} \right)}}} & {{Equation}\mspace{14mu}(1)}\end{matrix}$

When the BASg of the flat region 410 is higher than a predeterminedthreshold value, the flat region blocking artifact detector 110 maydetect that a blocking artifact is generated in the flat region 410 anddetermine that deblocking filtering needs to be performed on the flatregion 410.

FIG. 5 is a diagram for describing a method of removing a blockingartifact in a flat region, according to an exemplary embodiment. Anoperation of the flat region blocking artifact remover 120 is nowdescribed with reference to FIG. 5.

The flat region blocking artifact remover 120 may perform atransformation on pixels p1 through p4 of an input image 510 in blockunits. For example, the flat region blocking artifact remover 120 mayperform an 8×8 DCT on each of the blocks B1 through B4 having a size of8×8, wherein the pixels p1 through p4 are located at an upper leftcorner.

A DCT block group 520 includes DCT blocks CB1 through CB4 of the blocksB1 through B4, respectively. The DCT blocks are arranged according topixels of the input image 510. The flat region blocking artifact remover120 extracts three low frequency coefficients: a DC coefficient, a firstalternating current (AC) coefficient, and a second AC coefficient, fromamong transformation coefficients of the DCT blocks. For example, theflat region blocking artifact remover 120 may extract coefficients DC1,AC11, and AC12 from transformation coefficients of the DCT block CB1,and similarly, extract coefficients DC2, AC21, and AC22 of the DCT blockCB2, coefficients DC3, AC31, and AC32 of the DCT block CB3, andcoefficients DC4, AC41, and AC42 of the DCT block CB4.

The flat region blocking artifact remover 120 may classify the three lowfrequency coefficients, i.e., the DC coefficient, the first ACcoefficient, and the second AC coefficient, according to the coefficientlocation. The flat region blocking artifact remover 120 may form a lowfrequency coefficient image 531 in which the DC coefficients DC1 throughDC4 are arranged in an order of corresponding DCT blocks CB1 throughCB4. Similarly, a low frequency coefficient image 533, in which thefirst AC coefficients AC11 through AC41 are arranged in an order of DCTblocks, and a low frequency coefficient image 535, in which the secondAC coefficients AC12 through AC42 are arranged in an order of DCTblocks, may also be generated.

Since the low frequency coefficient images 531 through 535 are imagesincluding the low frequency coefficients of the input image 510, pixelsof the low frequency coefficient images 531 through 535 correspond tothe low frequency coefficients of the input image 510.

In operation 540, the flat region blocking artifact remover 120 mayremove the blocking artifact in the flat block by using the lowfrequency coefficient images 531 through 535. The flat region blockingartifact remover 120 may determine a smoothing filtering strength forremoving the blocking artifact in each flat block based on the local BASof the flat block. Low frequency coefficients of the low frequencycoefficient images 531 through 535 may be compensated for by performingdeblocking filtering based on an intrinsic smoothing filtering strengthdetermined according to the flat blocks with respect to the pixels i.e.,the low frequency coefficients corresponding to the flat blocks.

The flat region blocking artifact remover 120 may perform strongsmoothing filtering where the local BAS of a flat block is high, and mayperform weak smoothing filtering where the local BAS of a flat block islow.

FIG. 6 illustrates graphs for describing filtering for removing ablocking artifact in a flat region, according to an exemplaryembodiment.

As shown in FIG. 6 (a), brightnesses pA, pB, and pC of pixels A, B, andC are similar to each other and brightnesses pD, pE, and pF of pixels D,E, and F are similar to each other from among consecutive pixels (i.e.,low frequency coefficients) A through F in the low frequency coefficientimages 531 through 535 shown in FIG. 5. There is a brightness difference|pD−pC| between the pixel D and the pixel C with respect to a brightnessdifference d between the pixels A through C and the pixels D through F,and a block boundary is formed between the pixels A through C and thepixels D through F.

The flat region blocking artifact remover 120 may induce a filteringresult such as shown in FIG. 6 (b) when strong smoothing filtering isperformed on the block boundary between the pixels A through C and thepixels D through F. For example, the flat region blocking artifactremover 120 may perform filtering on the six consecutive pixels Athrough F based on the block boundary, from among the low frequencycoefficient images 531 through 535 shown in FIG. 5. Discontinuousbrightnesses pA through pF of the pixels A through F are respectivelycompensated for resulting in brightnesses pA* through pF*, and thus thebrightnesses of the pixels A through F may be evened so as to becontinuously and linearly increasing or decreasing.

When weak smoothing filtering is performed on the block boundary betweenthe pixels A through C and the pixels D through F, the flat regionblocking artifact remover 120 may compensate for the brightnesses ofsome pixels around the block boundary, such as shown in FIG. 6 (c). Forexample, the flat region blocking artifact remover 120 may performfiltering on four consecutive pixels B through E based on the blockboundary, from among the low frequency coefficient images 531 through535 shown in FIG. 5. Discontinuous brightness pB through pE of thepixels B through E are compensated for so as to result in brightnessespB** through pE**, and thus the brightnesses of the pixels A through Fmay be evened so as to continuously increase or decrease.

Here, a degree of compensation may be variably determined according to adistance between pixels to be compensated for and a block boundary. Forexample, the brightnesses pB** and pE** obtained by respectivelycompensating for the brightnesses pB and pE of the pixels B and E, thatare relatively far from the block boundary, have a relatively low degreeof compensation as a rate of increase from the brightness pB and pE is⅛. The brightnesses pC** and pD** obtained by respectively compensatingfor the brightnesses pC and pD of the pixels C and D, that arerelatively close to the block boundary, have a relatively high degree ofcompensation as a rate of increase from the brightnesses pC and pD is ½.

Accordingly, the flat region blocking artifact remover 120 may performstrong smoothing filtering (FIG. 6 (b)) where the local BAS of a flatblock is high, and weak smoothing filtering (FIG. 6 (c)) where the localBAS of a flat block is low.

Referring again to FIG. 5, the flat region blocking artifact remover 120may generate low frequency coefficient images 551 through 555 bycompensating for low frequency coefficients of the low frequencycoefficient images 531 through 535. The flat region blocking artifactremover 120 may select and extract low frequency coefficients in alocation corresponding to the transformation blocks of the input image510 from among low frequency coefficients of the low frequencycoefficient images 551 through 555.

Accordingly, the flat region blocking artifact remover 120 may extractone pixel per eight (i.e., ⅛) consecutive pixels in a locationcorresponding to a transformation block having a size of 8×8 of theinput image 510, from among the low frequency coefficients of the lowfrequency coefficient images 551 through 555, and replace a lowfrequency coefficient of the transformation block of the input image 510with the extracted pixel.

In other words, pixels extracted from the low frequency coefficientimage 551 may replace DC coefficients in the transformation blocks ofthe input image 510, pixels extracted from the low frequency coefficientimage 533 may replace the first AC coefficients in the transformationblocks of the input image 510, and pixels extracted from the lowfrequency coefficient image 555 may replace the second AC coefficientsin the transformation blocks of the input image 510.

For example, the flat region blocking artifact remover 120 may extractcompensation coefficients DC11 through DC44, one pixel per eight pixels,from the low frequency coefficient image 551. The flat region blockingartifact remover 120 may extract compensation coefficients AC111 throughAC441, one per eight pixels, from the low frequency coefficient image553, and extract compensation coefficients AC 112 through AC 442, oneper eight pixels, from the low frequency coefficient image 555.

Accordingly, DC coefficients of transformation blocks CB11 through CB44of an intermediate image 560 in a transformation region may berespectively replaced with the compensation coefficients DC11 throughDC44 extracted from the low frequency coefficient image 551. Similarly,the first AC coefficients of the transformation blocks CB11 through CB44of the intermediate image 560 may be respectively replaced with thecompensation coefficients AC111 through AC441 extracted from the lowfrequency coefficient image 553, and second AC coefficients of thetransformation blocks CB11 through CB44 may be respectively replacedwith the compensation coefficients AC112 through AC442 extracted fromthe low frequency coefficient image 555.

As a result, the DC coefficients, the first AC coefficients, and thesecond AC coefficients of the transformation blocks CB11 through CB44 ofthe intermediate image 560 are replaced with the compensationcoefficients of the low frequency coefficient images 551 through 555,but the other 61 coefficients (i.e., the 61 coefficients remaining outof the total of 64 coefficients) are maintained as coefficients of DCTblocks of the input image 510.

The flat region blocking artifact remover 120 may generate anintermediate image 570 in a spatial region by performing an inverse DCTon the intermediate image 560 in the transformation region in blockunits having a size of 8×8. The intermediate image 570 in a spatialregion is a resultant image obtained by removing a blocking artifact ina flat block from the input image 510.

FIG. 7 is a diagram for describing a method of removing a blockingartifact in an edge region, according to an exemplary embodiment. Amethod of removing a blocking artifact generated in an edge region of aninput image, which is performed by the edge region blocking artifactremover 140, is described with reference to FIG. 7.

The edge region detector 130 may generate a high frequency band image bysubtracting a low frequency band image including only a predeterminednumber of low frequency coefficients from transformation blocks of aninput image. The edge region detector 130 may determine a transformationblock of which the energy of samples is higher than a predeterminedthreshold value, from among transformation blocks of the high frequencyband image, as an edge block.

As shown in FIG. 7, the edge region blocking artifact remover 140 mayreceive an intermediate image 710 from which a blocking artifact in aflat region is removed by the flat region blocking artifact remover 120,and obtain locations of edge blocks from the edge region detector 130.The edge region blocking artifact remover 140 may determine edge blocksof the intermediate image 710 in correspondence to edge blocks detectedby the edge region detector 130. In the intermediate image 710, blockshaving solid line borders are edge blocks.

The edge region blocking artifact remover 140 may group edge blocks thatform the same edge, from among the edge blocks of the intermediate image710, in unit 720. For example, the edge region detector 130 maydetermine edge blocks 713 through 717 as forming the same edge bysearching for matching blocks 713 through 717 having bold solid lineborders from among adjacent blocks in a predetermined range based on anedge block (i.e., a reference block) 711 having a dashed-line border viablock matching for searching for a similar block.

Since the edge region blocking artifact remover 140 groups edge blocksby performing block matching on pre-detected edge blocks, throughput ofa 3D transformation technique based on block matching may be reduced.

The edge region blocking artifact remover 140 may determine at least onegroup of edge blocks forming the same edge. In other words, not only aGROUP #1 of the edge blocks 711 through 717 grouped via block matchingis not formed, but also various groups of edge blocks may be formed bysearching for edge blocks forming other edges.

For convenience of description, an operation of the edge region blockingartifact remover 140 is described as an operation for the GROUP #1, butthe operation may be performed for groups 1 through N, and results maybe combined.

The edge region blocking artifact remover 140 may calculate 3Dtransformation coefficients of edge blocks in a group by performing 3Dorthogonal transformation on the edge blocks 711 through 717 in theGROUP #1, and remove a blocking artifact in the edge blocks in a groupby performing threshold filtering on the 3D transformation coefficients,in unit 730.

According to a statistical distribution of 3D transformationcoefficients of an image, a possibility of a blocking artifactgeneration is low in a block in which distribution of 3D transformationcoefficients that are not “0” from among 3D transformation coefficients,is not dense. Accordingly, the edge region blocking artifact remover 140may reduce the number of 3D transformation coefficients that are not “0”by transforming a 3D transformation coefficient in an edge block lessthan a predetermined threshold value to “0” via threshold filteringaccording to a hard-thresholding technique for changing a 3Dtransformation coefficient to “0” if the 3D transformation coefficientis less than a predetermined threshold value. In other words, the edgeregion blocking artifact remover 140 may output edge blocks of which thedistribution of the 3D transformation coefficients that are not “0” isnot dense. Accordingly, a structure of edge blocks in a group ispreserved while transformation coefficients are compensated for, therebyremoving a blocking artifact in an edge block.

In unit 740, the edge region blocking artifact remover 140 may restorethe edge blocks 711 through 717 of GROUP #1 by performing an inverse 3Dorthogonal transformation on 3D transformation coefficients on whichthreshold filtering is performed. Also, in unit 740 the edge regionblocking artifact remover 140 may restore edge blocks by performing a 3Dorthogonal transformation, a threshold filtering, and an inverse 3Dorthogonal transformation for groups including GROUP #1 (and up to GROUP#N), and determine a weighted average block with respect to overlappingedge blocks from among the restored edge blocks according to groups,thereby updating an edge region of the intermediate image 710. As shown,in FIG. 7, the resulting final image may then be output. A blockingartifact that remains may be removed again per an edge block via theweighted average of the edge blocks according to groups.

FIG. 8 is a flowchart illustrating a method of removing an imageblocking artifact using a transformation coefficient, according to anexemplary embodiment.

The method according to the current exemplary embodiment may be realizedby a calculation processor, such as an image processing processor or aGPU.

In operation 810, a blocking artifact in a flat region is detected in aninput image. The input image may be an image decoded and restored by adecoder.

The blocking artifact in the flat region may be detected based on theBAS of flat blocks of the input image. The BAS according to an exemplaryembodiment may be determined based on a difference between DCcoefficients of adjacent blocks of a current flat block. Local BASes ofthe flat blocks are determined, whereas global BAS of the input image isdetermined based on an average of the local BASes, and when the globalBAS exceeds a predetermined threshold value, it may be determined thatthe blocking artifact is generated in the flat region of the inputimage.

In operation 820, the blocking artifact in the flat region is removed byusing a predetermined number of low frequency coefficients from amongtransformation coefficients according to blocks, for transformationblocks of the input image based on the result of detecting.

The blocking artifact in the flat region may be removed by compensatingfor the predetermined number of low frequency coefficients in thetransformation coefficients of blocks according to pixels of the inputimage. The predetermined number of low frequency coefficients may becompensated for according to those in the same coefficient location byclassifying the low frequency coefficients in the transformation blocksaccording to coefficient locations.

Smoothing filtering strength of the blocking artifact in the flat regionmay be determined by using the local BAS of the flat block determined inoperation 810, and smoothing filtering may be variably performedaccording to a location of the flat block.

The predetermined number of low frequency coefficients from among thetransformation coefficients according to blocks of the input image maybe replaced with the compensated for low frequency coefficients, andother transformation coefficients of the input image may remain as theyare. An image in a transformation region, in which part of the lowfrequency coefficients are replaced, is inverse transformed according totransformation blocks so as to generate an intermediate image in aspatial region from which a blocking artifact in a flat block isremoved.

In operation 830, an edge region is detected from the input image. Atransformation block of which the energy of samples is higher than apredetermined threshold value may be detected as the edge block fromamong transformation blocks of a high frequency band image of the inputimage.

Edge regions of the input image may be detected, prior to operations fordetecting and removing the flat region blocking artifact. In this case,blocking artifacts in flat regions of an image may be detected and,removed from regions which may not be the detected edge regions.

Flat regions and edge regions of the input image may be separatelydetected.

In operation 840, a blocking artifact in an edge region of anintermediate image is removed based on the detected edge region of theinput image, wherein the intermediate image is obtained by removing ablocking artifact in a flat region from the input image. A result imagefrom which the blocking artifact in the edge region is removed may beoutput after the blocking artifact in the flat region is removed fromthe input image.

Edge blocks forming the same edge are grouped from among edge blocks ofthe intermediate image, and 3D orthogonal transformation may beperformed on the edge blocks in a group to generate 3D transformationcoefficients of the edge blocks in a group. Threshold filtering isperformed and then inverse 3D orthogonal transformation is performed onthe 3D transformation coefficients of the edge blocks in a group toremove a blocking artifact in the edge blocks in a group. The edgeblocks of the intermediate image, on which the 3D orthogonaltransformation is performed and then the threshold filtering isperformed, may be updated by calculating a weighted average.

The apparatuses 100 and 300 according to the exemplary embodiments mayobtain a high quality digital television (DTV) image by effectivelyreducing a blocking artifact affecting the quality of an image viaquantized DCT coefficient compensation. Also, since various types ofencoding information is not used to remove an image blocking artifact, ablocking artifact may be effectively removed from uncompressed imagedata received through a D-sub terminal or Digital VideoInterface/High-Definition Multimedia Interface (DVI/HDMI) terminal of aTV.

Specifically, since the apparatuses 100 and 300 use a deblockingpost-processing technique that does not involve encoding information,wherein a hardware resource is not required to be added to an encodingand decoding system, the apparatuses 100 and 300 are useful in adeblocking technique on low transmission rate encoding image.

It would be interpreted by one of ordinary skill in the art that theblock diagrams described in the exemplary embodiments conceptuallyindicate a circuit for realizing principles. Similarly, it would beobvious to one of ordinary skill in the art that a predeterminedflowchart, a flow graph, a state transition diagram, and a pseudo codeare substantially expressed in a computer-readable recording medium andindicate various processes executed by a computer or a processor, evenif the computer or processor is not explicitly shown. Accordingly, theexemplary embodiments can be written as computer programs and can beimplemented in general-use digital computers that execute the programsusing a computer-readable recording medium. Examples of thecomputer-readable recording medium include magnetic storage media (e.g.,ROM, floppy disks, hard disks, etc.), optical recording media (e.g.,CD-ROMs, or DVDs), etc.

The functions of various elements shown in diagrams may be provided byusing not only hardware for executing software by being related tosuitable software, but also exclusive hardware. When the functions areprovided by a processor, the functions may be provided by a singleexclusive processor, a single common processor, or a plurality ofindividual processor, wherein some processors are shared. Also, terms“processor” or “controller” shall not be interpreted to exclusivelyindicate hardware for executing software, and may unlimitedly andimplicitly include digital signal processor (DSP) hardware, read-onlymemory (ROM) for storing software, random access memory (RAM), andnonvolatile storage devices.

Herein, an element expressed as a unit for performing a certain functionincludes a predetermined method of performing the certain function, andmay include a combination of circuit elements for performing the certainfunction, or software in a predetermined form including firmware ormicrocode combined to a suitable circuit for executing software forperforming the certain function.

In the present specification, “an exemplary embodiment” and othermodified expressions mean that a certain feature, structure, andcharacteristic are included in at least one exemplary embodiment.Accordingly, the expression “an exemplary embodiment” and other modifiedexamples in the present specification may not denote the same exemplaryembodiment.

In the present specification, the expression “at least one of A and B”is used to include a selection of only A, only B, or both A and B.Further, the expression “at least one of A through C” may be used toinclude a section of only A, only B, only C, only A and B, only B and C,or all of A through C. Similar expressions with more elements are alsopossible.

While exemplary embodiments have been particularly shown and described,it will be understood by those of ordinary skill in the art that variouschanges in form and details may be made therein. The exemplaryembodiments should be considered in a descriptive sense only and not forpurposes of limitation.

What is claimed is:
 1. A method of removing blocking artifacts in aninput image, the method comprising: detecting a first blocking artifactwhich is in a flat region of the input image; removing the firstblocking artifact using a number of low frequency coefficients fromamong transformation coefficients for each transformation block of aplurality of transformation blocks of the input image, based on a resultof detecting the first blocking artifact; detecting an edge region ofthe input image; and removing a second blocking artifact, which is in anedge region of an intermediate image obtained from the removing of thefirst blocking artifact of the input image, based on the detected edgeregion in the input image.
 2. The method of claim 1, wherein thedetecting the first blocking artifact comprises: determining flat blocksin the input image; and detecting the first blocking artifact based on ablocking artifact strength of the flat blocks.
 3. The method of claim 2,wherein the detecting the first blocking artifact based on the blockingartifact strength of the flat blocks comprises determining a currentblocking artifact strength of a current flat block based on a differencebetween direct current coefficients of the current flat block andadjacent blocks from among the flat blocks.
 4. The method of claim 2,wherein the detecting the first blocking artifact based on the blockingartifact strength of the flat blocks comprises: determining localblocking artifact intensities of the flat blocks; determining a globalblocking artifact strength of the input image based on an average of thelocal blocking artifact intensities of the flat blocks; and determiningthat the blocking artifact is generated in the flat region of the inputimage if the global blocking artifact strength exceeds a thresholdvalue.
 5. The method of claim 1, wherein the removing the first blockingartifact comprises: determining the transformation coefficientsaccording to pixels of the input image by transforming pixels in a blockunit including a current pixel and having a predetermined size, for eachof the pixels of the input image; and compensating for the number of lowfrequency coefficients from among the transformation coefficientsaccording to pixels, for the each of the pixels of the input image. 6.The method of claim 5, wherein the removing the flat block artifact inthe flat region comprises: replacing the transformation coefficientsaccording to blocks, which are obtained by dividing the input image inunits of the transformation blocks and transforming pixels in each ofthe transformation blocks, with the compensated for low frequencycoefficients; and generating an intermediate image in a spatial regionby inverse transforming the transformation coefficients of the inputimage according to the transformation blocks.
 7. The method of claim 5,wherein the compensating for the number of low frequency coefficientscomprises: generating low frequency coefficient images, in which lowfrequency coefficients from among the number of low frequencycoefficients which are in a same coefficient location are arranged in anorder of a corresponding block, for coefficient locations of the lowfrequency coefficients, by classifying the number of low frequencycoefficients according to coefficient locations in the transformationblocks; and removing the first blocking artifact for each of the lowfrequency coefficient images generated according to the coefficientlocations in the transformation blocks.
 8. The method of claim 6,wherein the replacing the transformation coefficients according toblocks, which are obtained by dividing the input image in block unitswith the compensated for low frequency coefficients, comprises:extracting low frequency coefficients of blocks in a same location asthe transformation blocks of the input image, from among the compensatedfor low frequency coefficients; and replacing low frequency coefficientsin a same location as the extracted low frequency coefficients, fromamong the low frequency coefficients of the transformation blocks of theinput image, with the extracted low frequency coefficients.
 9. Themethod of claim 4, wherein the removing the first blocking artifactcomprises determining a smoothing filtering strength for removing thefirst blocking artifact, for each of the flat blocks, based on the localblocking artifact intensities of the flat blocks, and removing the firstblocking artifact based on the determined smoothing filtering strength.10. The method of claim 4, wherein the removing the first blockingartifact comprises determining a filtering strength of total variationfiltering for removing the first blocking artifact based on the localblocking artifact intensities of the flat blocks, and removing the firstblocking artifact by performing the total variation filtering based onthe determined filtering strength.
 11. The method of claim 1, whereinthe detecting the edge region in the input image comprises: generating alow frequency band image by inverse transforming an image that includesthe number of low frequency coefficients for transformation blocks ofthe input image, wherein other coefficients are 0; and generating a highfrequency band image by subtracting the low frequency band image fromthe input image.
 12. The method of claim 11, wherein the detecting theedge region in the input image further comprises determining atransformation block of which the energy of samples is higher than athreshold value from among transformation blocks of the high frequencyband image, as an edge block.
 13. The method of claim 12, wherein theenergy of samples is an average of absolute values of the transformationcoefficients of the transformation blocks or an average of absolutevalues of pixel values of the transformation blocks.
 14. The method ofclaim 1, wherein the removing the second blocking artifact comprises:grouping edge blocks forming a same edge, from among edge blocks of theedge region of the intermediate image, in correspondence to the edgeregion of the input image; performing a three-dimensional (3D)orthogonal transformation on the edge blocks in each group resultingfrom the grouping; and removing the second blocking artifact by using 3Dtransformation coefficients of the edge blocks in each group resultingfrom the grouping.
 15. The method of claim 14, wherein the removing ofthe blocking artifact of the edge blocks using the 3D transformationcoefficients of the edge blocks in each group comprises performingthreshold filtering for transforming the 3D transformation coefficientsto 0 if the 3D transformation coefficients are less than a thresholdvalue.
 16. The method of claim 14, wherein the removing of the blockingartifact of the edge blocks using the 3D transformation coefficients ofthe edge blocks in each group comprises: performing threshold filteringon the 3D transformation coefficients; restoring the edge blocks in eachgroup by performing an inverse 3D orthogonal transformation on the 3Dtransformation coefficients on which the threshold filtering isperformed; and updating the edge region detected in the intermediateimage based on the restored edge blocks.
 17. The method of claim 16,wherein the updating of the edge region detected in the intermediateimage based on the restored edge blocks comprises updating the edgeregion by determining a weighted average block with respect to edgeblocks in an overlapping region from among edge blocks restored byperforming the 3D orthogonal transform, the threshold filtering, and theinverse 3D orthogonal transformation according to a plurality of groups.18. The method of claim 1, further comprising receiving the input image,wherein the input image is decoded and restored by a decoder.
 19. Anapparatus for removing image blocking artifacts in an input image, theapparatus comprising: a flat region blocking artifact detector thatdetects a first blocking artifact that is in a flat region of the inputimage; a flat region blocking artifact remover that removes the firstblocking artifact by using a number of low frequency coefficients fromamong transformation coefficients according to blocks, for each oftransformation blocks of the input image based on a result of thedetection by the flat region blocking artifact detector; an edge regiondetector that detects an edge region in the input image; and an edgeregion blocking artifact remover that removes a second blockingartifact, which is in an edge region of an intermediate image, obtainedby removing the first blocking artifact of the input image, based on thedetected edge region in the input image.
 20. The apparatus of claim 19,wherein the flat region blocking artifact detector determines flatblocks in the input image, and detects the first blocking artifact basedon a blocking artifact strength of the flat blocks.
 21. The apparatus ofclaim 20, wherein the flat region blocking artifact detector determinesa current blocking artifact strength of a current flat block based on adifference between direct current coefficients of the current flat blockand adjacent blocks from among the flat blocks.
 22. The apparatus ofclaim 20, wherein the flat region blocking artifact detector: determineslocal blocking artifact intensities of the flat blocks; determines aglobal blocking artifact strength of the input image based on an averageof the local blocking artifact intensities of the flat blocks; anddetermines that the blocking artifact is generated in the flat region ofthe input image if the global blocking artifact strength exceeds athreshold value.
 23. The apparatus of claim 19, wherein the flat regionblocking artifact remover determines transformation coefficientsaccording to pixels of the input pixels by transforming pixels in ablock unit including a current pixel and having a predetermined size,for each of the pixels of the input image, and compensates for thenumber of low frequency coefficients from among the transformationcoefficients for the each of the pixels of the input image.
 24. Theapparatus of claim 23, wherein the flat region blocking artifact removerreplaces the transformation coefficients according to blocks with thecompensated for low frequency coefficients, for the transformationblocks of the input image, and generates an intermediate image in aspatial region by inverse transforming the transformation coefficientsof the input image for each of the transformation blocks.
 25. Theapparatus of claim 23, wherein the flat region blocking artifact removergenerates low frequency coefficient images, in which low frequencycoefficients from among the number of low frequency coefficients whichare in a same coefficient location are arranged in an order of acorresponding block, for coefficient locations of the low frequencycoefficients, by classifying the number of low frequency coefficientsaccording to coefficient locations in the transformation blocks, andremoves the first blocking artifact in flat blocks for each of the lowfrequency coefficient images generated according to the coefficientlocations.
 26. The apparatus of claim 24, wherein the flat regionblocking artifact remover extracts low frequency coefficients of blocksin a same location as the transformation blocks of the input image, fromamong the compensated for low frequency coefficients, and replaces lowfrequency coefficients in a same location as the extracted low frequencycoefficients, from among the low frequency coefficients of thetransformation blocks of the input image, with the extracted lowfrequency coefficients.
 27. The apparatus of claim 22, wherein the flatregion blocking artifact remover determines a smoothing filteringstrength for removing the blocking artifact according to the flat blocksbased on the local blocking artifact intensities of the flat blocks, andremoves the first blocking artifact based on the determined smoothingfiltering strength.
 28. The apparatus of claim 22, wherein the flatregion blocking artifact remover determines a filtering strength oftotal variation filtering for removing the first blocking artifact basedon the local blocking artifact intensities of the flat blocks, andremoves the first blocking artifact by performing the total variationfiltering based on the determined filtering strength.
 29. The apparatusof claim 19, wherein the edge region detector generates a low frequencyband image by inverse transforming an image that includes the number oflow frequency coefficients for transformation blocks of the input image,and wherein other coefficients are 0, and generates a high frequencyband image by subtracting the low frequency band image from the inputimage.
 30. The apparatus of claim 29, wherein the edge region detectordetermines a transformation block, of which the energy of samples ishigher than a predetermined threshold value from among transformationblocks of the high frequency band image, as an edge block.
 31. Theapparatus of claim 30, wherein the energy of samples is an average ofabsolute values of the transformation coefficients of the transformationblocks or an average of absolute values of pixel values of thetransformation blocks.
 32. The apparatus of claim 19, wherein the edgeregion blocking artifact remover groups edge blocks forming a same edge,from among edge blocks of the edge region of the intermediate image, incorrespondence to the edge region of the input image, performs athree-dimensional (3D) orthogonal transformation on the edge blocks ineach group grouped by the edge region blocking artifact remover, andremoves the blocking artifact of the edge blocks by using 3Dtransformation coefficients of the edge blocks in each group grouped bythe edge region blocking artifact remover.
 33. The apparatus of claim32, wherein the edge block blocking artifact remover performs thresholdfiltering for transforming the 3D transformation coefficients to 0 ifthe 3D transformation coefficients are less than a threshold value. 34.The apparatus of claim 32, wherein the edge block blocking artifactremover performs threshold filtering on the 3D transformationcoefficients; restores the edge blocks in each group by performing aninverse 3D orthogonal transformation on the 3D transformationcoefficients on which the threshold filtering is performed, and updatesthe edge region detected in the intermediate image based on the restorededge blocks.
 35. The apparatus of claim 34, wherein the edge blockblocking artifact remover updates the edge region by determining aweighted average block with respect to edge blocks in an overlappingregion from among edge blocks restored by performing the 3D orthogonaltransform, the threshold filtering, and the inverse 3D orthogonaltransformation according to a plurality of groups.
 36. The apparatus ofclaim 19, wherein the input image is decoded and restored by a decoder.37. A computer-readable recording medium having recorded thereon aprogram for executing the method of claim
 1. 38. A method of removingblocking artifacts in an input image, the method comprising: detectingat least one first blocking artifact, which is in a flat region of theinput image, based on a first threshold comparison; transforming blocksof the input image, wherein the transforming generates transformationcoefficients for each transformed block of the input image; extractinglow frequency coefficients from among the transformation coefficientsgenerated from the transforming; generating low frequency coefficientimages based on the extracted low frequency coefficients; generatingcompensated low frequency coefficient images by filtering the at leastone first blocking artifact from the generated low frequency coefficientimages; extracting compensation coefficients from the compensated lowfrequency coefficient images; and generating an intermediate image byinverse transforming the compensated low frequency coefficient images.39. The method of claim 38, further comprising: detecting at least oneedge block in the intermediate image based on a second thresholdcomparison; performing a three-dimensional (3D) transformation on groupsof edge blocks, which form a same edge, from among the at least one edgeblock detected in the intermediate image; filtering 3D transformationcoefficients resulting from the 3D transformation; performing an inverse3D transformation based on a result of the filtering of the 3Dtransformation coefficients; and generating a final image based on theperforming of the inverse 3D transformation.